On interim rationality, belief formation and learning in decision problems with bounded memory∗

We consider decision problems where the agent has bounded memory and makes decisions using a finite state automaton. For any automaton, we show there exist equivalent automata with minimal randomization. We introduce a new criterion for interim rationality called weak admissibility and demonstrate that if memory states are not wasted, the nature of transitions are uniquely defined by the criterion. We show for any optimal automaton, there exists an equivalent automaton that is weakly admissible and involves no wastage of memory states. We describe more general environments (POMDP) and establish a revelation principle for optimality. Under a changing worlds environment, we discuss conditions under which unconstrained optimum can be attained using finite state automata. In these environments, we demonstrate characteristics of the constrained optimal automaton.